Credal Classification based on AODE and compression coefficients

Bayesian model averaging (BMA) is a common approach to average over alternative models; yet, it usually gets excessively concentrated around the single most probable model, therefore achieving only sub-optimal classification performance. The compression-based approach (Boullé, 2007) overcomes this problem; it averages over the different models by applying a logarithmic smoothing over the models’ posterior probabilities. This approach has shown excellent performances when applied to ensembles of naive Bayes classifiers. AODE is another ensemble of models with high performance (Webb et al., 2005): it consists of a collection of non-naive classifiers (called SPODE) whose probabilistic predictions are aggregated by simple arithmetic mean. Aggregating the SPODEs via BMA rather than by arithmetic mean deteriorates the performance; instead, we propose to aggregate the SPODEs via the compression coefficients and we show that the resulting classifier obtains a slight but consistent improvement over AODE. However, an important issue in any Bayesian ensemble of models is the arbitrariness in the choice of the prior over the models. We address this problem by adopting the paradigm of credal classification, namely by substituting the unique prior with a set of priors. Credal classifier are able to automatically recognize the prior-dependent instances, namely the instances whose most probable class varies, when different priors are considered; in these cases, credal classifiers remain reliable by returning a set of classes rather than a single class. We thus develop the credal version Corresponding author: [email protected] Preprint submitted to Elsevier March 28, 2012 of both the BMA-based and the compression-based ensemble of SPODEs, substituting the single prior over the models by a set of priors. By experiments we show that both credal classifiers provide overall higher classification reliability than their determinate counterparts. Moreover, the compression-based credal classifier compares favorably to previous credal classifiers.